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GAUSSIAN  iROBABILiry  MEASURE  DEFINED  BY  A  GENERALIZED  RELATIVE  DENSITY 

ABSTRACT 

A  generalized  gaussian  relative  probability  density  is  introduced 
in  an  infinite  dimensional  linear  space.  Although  it  is  zero  with 
probability  one,  it  achieves  to  describe  an  actual  probability  distri¬ 
bution  in  another  space  by  the  behavior  of  its  maxima  on  linear 
varieties.  The  calculus  of  random  functions  in  terms  of  variational 
problems,  which  suggested  this  introduction,  is  to  be  the  subject  of 
subsequent  reports. 
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An  elementary  property  of  gaussian  probability  densities  in  several 
variables  is  seen  to  motivate  vhat  must  surely  be  the  simplest,  most 
direct  and  intuitive  approach  possible  to  gaussian  probability  distri¬ 
butions  in  infinite  dimensional  linear  spaces.  A  gaussian  distribution 
is  described  by  means  of  a  generalized  relative  probability  density, 
the  obvious  formal  generalization  of  the  gaussian  density  in  several 
variables.  It  cannot,  of  course,  be  the  probability  per  unit  volume 
since  there  is  no  volume.  Rather  the  distribution  is  described  in 
terms  of  the  behavior  of  the  maxima  of  this  generalized  density  on 
linear  varieties.  The  elementary  property  which  motivates  the  treatment 
turns  out  to  be  just  what  is  needed  to  fulfill  the  hypothesis  of  a 
f\mdamental  theorem  of  Kolmogorov  [IJ  which  assures  the  existence,  in 
any  of  certain  larger  spaces,  of  probability  measures  so  described. 

The  equivalence  of  these  measures  is  then  inferred  without  difficulty 
and  perhaps  contributes  a  bit  to  an  understanding  of  the  gaussian 
distribution. 

The  densities  will  be  studied  further  and  applications  will  be 
made  in  subsequent  reports . 

At  this  late  date  no  introduction  to  gaussian  distributions  is 
apt  to  be  totally  new.  Although  the  present  treatment  seems  rather 
novel  (to  the  author  at  least it  differs  from  many  existing  dis¬ 
cussions  principally  in  that  it  avoids  the  use  of  a  preferred  repre¬ 
sentation  of  the  elements  of  the  linear  space.  The  maximum  principle 
on  which  it  is  based  has  been  used  repeatedly  by  Feynman  fl,  2j  and 
probably  others  in  the  evaluation  of  individual  functional  Integrals. 

It  is  pleasant  to  note  that  the  space  in  which  the  density  is  defined 
need  not  be  separable. 

N 

1.  In  the  space  R  of  N-tuplets  u:  vu,  u^,  .  .  .  ,  u^  of  real 
numbers,  a  gaussian  probability  distribution  is  defined  by  a 
probability  density 
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<p(i.)  =  k  exp  (-  I  J(u)) 


(1) 


where  j(u)  Is  a  positive  definite  quadratic  polynomial,  a  translation 
of  a  positive  definite  quadratic  form  K(u); 

J(u)  =  K(u  -  u^)  (2) 

for  some  u^  e  R^.  In  (l),  k  is  a  normalizing  constant  which  will 
henceforth  be  omitted.  The  resulting  relative  gaussian  probability 
density  will,  for  brevity,  be  described  as  a  density.  The  values 
v:  Vj^^  .  .  .  ,  Vj^,  M  i  N,  of  any  set  of  Independent  linear  combi¬ 
nations  of  the  u*s, 

N 

i5(u):  i^(u)  =  Uj,  i  =  1,  2,  .  .  .  M,  (3) 

are  random  variables  whose  simiiltaneoiis  distribution  is  described  by 
a  density  'l'(v)  which  may  be  found  by  integrating  out  the  excess 
variables  in  (2).  However,  it  may  also  be  obtained  simply  as  the 
maximvim  of  (p(u)  taken  subject  to  the  condition  that  the  linear 
functions  i(u)  take  the  specified  valixes: 

Theorem  1  If  u  is  a  set  of  gaussian  random  variables 

having  a  density  <p(u),  the  density  of  the  values  v  of 
independent  linear  combinations  (3)  is 

\|r(v)  =  max  ((jp(u)  :  i(u)  =  v) 

=  exp  (-  -  min  (j(u)  :  i(u)  =  v)).  (4) 
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This  resvilt  becomes  apparent  when  a  linear  transformation  is 
made  to  independent  variables  of  which  the  first  M  are  linear 
combinations  of  i^(u),  .  .  .  , 

So  far  as  the  author  knows,  this  simple  property  of  normal 
distributions  has  not  before  been  expressly  stated.  Here  it  will 
serve  first  to  motivate  the  definition  of  a  generalized  density  in 
an  infinite  space,  and  second,  to  provide  a  principal  cog  in  the 
mechanism  by  which  the  generalized  density  is  seen  to  describe  a 
probability  distribution  in  a  different  space,  a  certain  second 
conjugate  space. 

2.  Let  S  be  a  real  linear  space  in  which  there  is  defined  an 
inner  product  (u,  v),  u,  v  e  S> which  is  positive  definite,  i.e., 

(u,  u)  =  ||u|p  >0,  if  u  0.  (5) 

Such  a  space  S  may  be  completed  to  form  the  mique  minimal  hilbert 
space  SJ  S.  A  positive  definite  quadratic  functional  J(u),  u  e  S, 
is  a  functional  which  can  be  written  in  terms  of  such  an  inner 
product  as 

J(u)  =  +  '^(u)  +  const  (6) 

where  i(u)  is  a  linear  functional  bounded  with  respect  to  |u|  . 

The  definition  of  the  functionals  j(u)  and  i(u)  may  be  extended 

continuously  to  S  in  a  unique  way.  The  representation  theorem 

assTures  the  existence  of  an  element  u  e  S  such  that 

o 

i(u)  =  -  |(u^,  u)  (7) 

for  all  u  €  S.  Then,  using  the  linearity  of  the  inner  product. 
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J(u)  =  Ju 


2 


+  const . 


(8) 


A  functional 

(p(u)  =  exp  -  I  J(u),  u  e  S,  (9) 

where  J(u)  is  a  positive  definite  qtiadratic  functional  will  he 
called  a  generalized  relative  gaussian  probability  density,  or 
briefly,  a  density.  It  will  be  the  aim  of  the  following  paragraphs 
to  justify  this  terminology. 

Let  S*  be  the  normed  conjugate  space  of  S,  consisting  of  all 
bounded  linear  functionals  defined  on  S.  To  each  v*(u)  e  S* there 
corresponds  v  e  S  such  that  v* (u)  =  (v,  u)  for  al  l  u  e  S  (or  S). 

This  correspondence,  in  which  both  norm  and  linear  structure  are 
invariant,  makes  it  frequently  unnecessary  to  distinguish  between 
a  hilbert  space  and  its  conjugate.  Here,  however,  the  reader  may 
find  it  helpfxil  to  emphasize  the  distinction.  Further,  let 
be  the  algebraic  conjugate  space  of  S‘,  consisting  of  the  linear 
functionals,  u  (u‘ ),  not  necessarily  bounded,  defined  on  S*. 

For  these  functionals  the  following  extensions  of  earlier  notation 
will  be  used: 


«(v’)  -  v-(«)  =  (v,  «).  (10) 


The  space  S  will  be  the  space  in  which  is  to  be  defined  the  proba¬ 
bility  measure  described  by  (9).  To  each  element  v'  of  S*  will 
then  be  associated  a  random  variable  (lO) . 

Any  finite  set  u^^*  ,  U2*  ,  .  •  ■  .  Uj^'  of  linearly  independent 
elements  of  S'  has  an  N-dimensional  probability  distribution 
described  by  a  density  t(w^,  w^,  .  .  .  .  ,  w^^)  in  the  ordinary 


8 


sense  that  if  E  is  any  measurable  set  of  the  probability  that 

the  N-tuplet  :  vi,*(u),  u_Vu) . u  Upb  in  in  in 

X  '  '  <  ■  '  “N  - —  “ 


E 


f^(y) 


Vr 


/n  \ 


The  density  is  given  in  analogy  with  (4)  by  the  conditioned  supremum 


'l'(v)  =  1,  t  , 

-1  ,  Ug  , 


..  «  (w, , 
.  •  X- 


.  .  w..) 
'  JN' 


=  sup  ((D(u):u  e  S,  u. ‘fu)  =  w.  .  i  =  1.2. 
,  -  -  -  1  '  '  1' 


=  exp  -  I  inf  (J(u):  u  e  S,  u^‘(u)  =  w^,  i  =  1,2,  .  ..,  N).(l2) 

The  sun'reTniiTn  nviv.  nf*  r*rkii>*Qo  "Kq  vi^-rxi » _ ^  ^  n  ^  ^  _ i 

--c- - — j  X  1^^043, uj  luckjLxiuuui  XX  Q  ib  rep_Laced 

by  S.  In  order  to  verify  that  ^^(w),  so  defined  is  a  gaussian  density, 
it  must  be  shown  the  exponent  which  occurs  in  (12)  is  a  definite 
Quadratic  uolvnomlal,  "M  OTM^n+e  tr  ^  O  4  _  TO  nr 

-  X  w -  - - -  w  *,  ^  u,  X  —  •••  A>l, 

by  means  of  which  it  is  possible  to  express  the  functionals  u^*  as 
inner  products : 


(u)  =  (v^,  u)  for  all  u  e  S. 


(13) 


Any  element  u  e  S  ceui  be  expressed,  with  real  a., ,  .  .  .  ,  a. 

i 

in  the  form 


u  =  u  + 
o 


N 

E 

t=i 


^  ^i 


(14) 


wnere 


fl  ttnil  (m  .  =  0-  1  sal.  P. 


TVT 
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Then  the  conditions  on  the  infimum  in  (12)  can  be  written 


Wj  =  (Vj,  u) 


l=i  “ 


J=1.2, 


N. 


(16) 


The  statement  that  thp  v  . 

- 


'N 


J- J-Xi^caj.  J-Jr  iAAUCpcXlLLCllU  xa 


equivalent  to  the  non-vanishing  of  the  determinant  of  the  quantities 
i*J  =  1>  •  •  •  >  N;  and  it  is  seen  from  (l6)  that  speci¬ 
fying  (*v..  u).  .1  =  1.  .  . 

-  -  •  j' 


N.  *1  «  omi-f  tro  T  ca»-»4*  -t-z-x  -P-.f-l  «i 

7  7  W  \jyj  jr  XXl^ 


"l' 


Since 


J(u)  =  |u-u^|2  = 

JL  . 

=  2  (^i'  ^1^  ^  (17) 

o  —  o 

i,J=l 


it  is  clear  that  the  infimum  of  j(u)  with  the  a's  specified  is 
obtained  by  setting  r  =  0,  and  that  the  value  so  obtained  is  positive 
and  is  a  quadratic  form  in  a^^,  .  .  .  ,  a^,  or,  equivalently,  in 


Finally,  in  order  to  justify  the  term  generalized  density,  it 
must  be  shown  that  there  exists  a  probabilitv  mpfl  flnr»P  rk-P  \j‘h^  oVi  -♦-■hck 

XT" - 1/  — ~  WXXV^ 

^x.x.«.jr  wx  iucxxe,j-iAcx-L  u.ciiDxuxea  uuotixiiea  irom  is  a  proper  description. 

/ 

This  results  from  a  straightforward  application  of  the  fundamental 
theorem  of  Kolmogorov  fl,  Ch.  ml  : 

M 

Let  M  be  any  set.  Let  R“  be  the  space  of  real  functions  x 
defined  before  all  u  e  M.  If  for  every  finite  subset  u  .  u  .  .. 

^1,  .  r-j, 

OT  M  ^  G  rr-f  TrciKk  o  V « V. -t  T  4  X , >3  4,^4 4T J_  4 j  « 

--  ..  w  0-.-»v,xx  t*  ^Xi^ucxuxxx  LLXCa  UX'XUUUXUIl  XH  Xlie  SpaCC  OT 


the  values  of  x 


,  X  ,  €Lnd  if  these  finite  dimensional 


^1  "2  ‘"H 

distributions  are  compatible,  then  there  exists  a  countably  additive 

M 

probability  measure  in  R  defined,  over  the  field  of  those  subsets 

of  countably  generated  by  half  spaces  (sets  defined  for  some 

U  €  M  and  real  k,  by  x^^  k)  and  having  the  given  distributions  as 

marginal  distributions,  i.e.,  as  distributions  of  sets  x  .....  x 

a’ 

M 

considered  as  random  variables  on  R  , 


In  applying  this  theorem  to  the  present  situation  the  index  set  M 
may  be  taken  as  a  Hamel  basis  Z  of  S',  that  is,  a  finitely  linearly 
independent  subset  of  S’  such  that  every  element  of  S'  can  be  expressed 
in  a  unique  way  as  a  finite  linear  combination  of  elements  of  Z.  The 
compatibility  condition,  that  the  given  lower  dimensional  distributions 
are  obtained  in  every  case  by  integrating  out  the  excess  variables  from 
the  given  higher  dimensioneil  distributions,  is  here  immediately  seen  to 
be  satisfied  by  virtue  of  Theorem  1  and  the  fact  that,  for  ^  N, 


max 

^N^+1’  ••• 


sup  (<p  (u):u^'(u)  =  k^,  i  =  1,  2, 
kj^  ueS 


N) 


=  sup  (q) (u):u^* (u)  =  k^,  i  =  1,  2,  . . .  ,  N^).  (l8) 

ueS 

The  conclusion  may  now  be  drawn  that  the  generalized  density  describes 
a  probability  measure  on  the  space  of  functionals  on  Z.  But  since  Z 
is  a  Hamel  basis  of  S’ ,  the  linear  fmctionals  on  S  ’  are  one-to-one 
linear  images  of  the  functionals  on  Z.  The  generalized  density  may 
thus  equally  well  be  said  to  describe  a  probability  measure  on  It 
is  also  clear  that  this  measure  is  independent  of  the  choice  of  the 
Hamel  basis  Z.  This  completes  the  proof  of 
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Theorem  2;  A  generalized  gauaslan  density  (9)  defined 
on  a  linear  space  S:  describes  by  means  of  the 

marginal  densities  (12)  a  vuiique,  countably  additive 
probability  measure  on  the  space  S  of  linear  fionctionals 
defined  on  the  normed  conjugate  space  S'  of  S,  con¬ 
sisting  of  those  linear  functionals  on  S  which  are 
bounded  with  respect  to  the  nonn  provided  by  the  given 
density.  This  measure  is  defined  over  the  borel  field 

/V 

of  sets  of  S  generated  comtably  by  half  spaces,  the 
sets  £u  e  S  u(u*  k^u'eS*,  k  real. 

3.  Either  as  a  consequence  of  Theorem  2,  or  more  simply  by 
repeating  the  argument  with  minor  amendments  the  following  corollary 
is  obtained. 

CoTOl^^:  Let  V*  be  a  subset  of  S'  and  ^  the  space 
of  linear  functionals  defined  on  V.  The  generalized 
density  (9)  describes  a  unique,  countably  additive 
probability  measiare  in  0  over  the  borel  field  of  sets 
of  ^  generated  by  half  spaces 
V*  eV*,  k  real. 

If  V*  is  co^lete  in  S%  the  measure  defined  in  ^  is  an  adequate 
representation  of  the  measure  defined  in  S  in  the  sense  described  in 
the  following  theorem.  For  the  statement,  there  is  needed  the 
definition  that  two  sets  are  equivalent  if  their  differences  are  of 
measure  zero. 

Theorem  If  the  set  V*  is  complete  in  S*,  the  measures 
-  A  A 

defined  on  S  and  V  are  isomorphic  in  the  sense  that  there 

V 

Is  a  one-to-one  correspondence  between  classes  of  equivalent 
sets  in  the  two  spaces  in  which  measure,  inclusion  emd 
translation  by  elements  of  S  are  preserved. 
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A  A 

Proof:  There  is  a  projection  of  S'  into  V  in  which  each  element  of 
a  linear  functional  on  S*,  is  carried  into  its  restriction  to  V* . 
To  each  measurable  set  E._  of  V  let  correspond  the  set  of  §  con- 

sisting  of  all  elements  which  are  projected  into  elements  of  E^. 

A 

A  sequence  of  operations  on  the  half  spaces  of  V  which  defines  E^^ 

-  -  -  y 

defines  similarly  when  applied  to  the  corresponding  half  spaces  of 

A 


S,  a  set  of  Eq  of  the  same  measure.  There  remains  only  the  question 
whether  every  measurable  set  of  S  is  included  in  this  correspondence 
and  it  will  suffice  to  show  that  every  half  space  is  included,  or, 
in  other  words,  to  show  that  to  each  bounded  linear  functional 

A 

u'eS  there  is  an  equivalent  random  variable  defined  on  V.  Since 
V*  is  complete  in  S',  there  is  a  sequence  v^* ,  v^'  ,  ...  in  V'  such 
thatjjv^*  -  u'jj  tends  to  zero.  An  immediate  consequence  of  (12) 
is  that  the  norm  of  any  bounded  linear  functional  on  S  is  the 
standard  deviation  of  the  random  variable  it  defines.  The  sequence 


^Vi'|-must  therefore  be  regular  in  probability  in  V  and  defines  (see 
e.g.,  Halmos  [1]  Sec.  22,  Theorem  E,  p.  93)  in  V  a  random  variable 
equivalent  to  u*  (ibid..  Theorem  C).  This  completes  the  proof. 


Perhaps  it  should  be  emphasized  that  this  isomorphy  is  not  a 
mapping  of  elements.  If  the  ST)ace  is  Infinite  and  if  is  a  small 
(compared  to  a  Hamel  basis)  set  then  it  will  not  be  possible  to  express 

A 

any  significant  portion  of  the  elements  of  S  as  linear  images  of 
elements  of  If  S  is  a  separable  space,  then,  as  Friedrichs  and 

Shapiro  have  shown,  linear  mappings  of  elements  can  be  produced 

which  work  with  probability  one  and  provide  a  realization  of  the 
isomorphy  of  the  measure  spaces  related  by  a  change  of  coordinates. 


/IJ  /7) 


WILLIAM  C.  TAYLOR 
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